1. Field of the Invention
The present invention relates to an image processing apparatus and an image processing method that reduce noise in sequentially inputted images.
2. Description of the Related Art
Video signals represent a plurality of frame (field) images picked up in a specified time period. In general, the video signals are supposed to represent a correlation within a local space of the frame (Field) by each frame (field) image and represent a correlation within a local time between frames (fields) by adjacent frame (field) images.
Video signals can be obtained by picking up an image of a subject with a video camera provided with an image pickup device, such as a CCD and a CMOS. For example, a digital video camera obtains digital video signals as follows. First, a lens of an image pickup section of the video camera forms an optical subject image on an image pickup device. The formed optical subject image is photoelectrically converted into an electrical signal for each pixel on the image pickup device. The electrical signals are outputted from the image pickup device in a predetermined order. The signals outputted from the image pickup device are analog image signals. The analog image signals are amplified with a predetermined gain and then converted into digital image signals by an A/D converting section. Then, in general, a digital signal processing circuit performs a predetermined image processing on the digital image signals thus obtained.
In general, noise is superimposed on the image picked up by the video camera. The noise has many causes, one of which is characteristics of the image pickup device. A representative one of the noise caused by the characteristics of the image pickup device is shot noise that is caused by a statistical characteristics involved in photoelectric conversion. The shot noise has an average amplitude proportional to the square root of the image signal value and, in general, is temporally and spatially statistically random.
Although the random noise is literally “random”, the video signals have a local spatial correlation and a local temporal correlation as described above. Thus, to reduce the random noise superimposed on the video signals, an intra-frame (intra-field) noise reduction processing based on the spatial correlation and an inter-frame (inter-field) noise reduction processing based on the temporal correlation can be used. Various types of noise reduction processing techniques that can be applied to the video signals have been already proposed.
Unless otherwise specified, one picked up image will be referred to as a frame in the following description. And the present invention can be equally applied even if the term “frame” is replaced with the term “field” in the following description.
Of the two types of noise reduction processings described above, the type of noise reduction processing based on the temporal correlation includes a recursive noise reduction processing that uses a frame reduced in noise as a previous frame. The recursive noise reduction processing is known as a technique that can more effectively reduce noise than many other noise reduction processings.
For example, Japanese Patent Application Laid-Open Publication No. 2-184176 describes a technique which achieves noise reduction by detecting a variation of a video for each screen section, calculating a motion signal proportional to the variation, and mixing the current video signal, at a predetermined ratio, with a signal that is selectively weighted with either a weight applied to a video signal of the previous frame or a weight applied to a video signal in the previous line of the current frame depending on the motion signal.
Further, a noise reduction technique that uses both the intra-frame correlation and the inter-frame correlation to improve noise reduction for a dynamic scene is described in Japanese Patent Application Laid-Open Publication No. 6-62283, for example. A noise reduction system described in the publication has an image memory to provide a delay of one frame or one field. The noise reduction system outputs pixel data reduced in noise by performing a nonlinear filtering processing on center pixel data about a center pixel and nearby pixel data about a pixel close to the center pixel that are newly inputted thereto and nearby pixel data about a pixel close to the center pixel in image data for the previous frame or field already reduced in noise and recorded in the image memory. The nonlinear filter is designed to take a weighted average by assigning a high weighting factor to nearby pixel data having a high correlation with the center pixel data value and a low weighting factor to a pixel having a low correlation. Thus, noise can be reduced by using both the intra-frame correlation and the inter-frame correlation. In particular, when the picked up image is static, the technique can effectively reduce noise because the pixels in the previous frame or field and pixels in the current field used for the weighted averaging include an increased number of pixels having high weighting factors, and thus an increased number of pixels contribute to the averaging. When the picked up image is dynamic, pixels in the current field have higher weighting factors than pixels in the previous frame or field, so that the effect of the pixels in the current frame is predominant in the weighted averaging, and thus, the amount of noise reduced decreases compared with the case of the static image. However, the noise reduction effect is still higher than the simple recursive noise reduction processing based on the pixels at the same position in the current frame and the previous frame.
In Japanese Patent Application Laid-Open Publication No. 6-350879, another noise reduction technique is described. The technique performs an inter-frame noise reduction processing by calculating a correlation between a predetermined block in the current frame and blocks of image signals in the previous frame by block matching, extracting a block determined as having the highest correlation from the previous frame, and performing a noise reduction processing on the extracted block and the predetermined block in the current frame. The technique can effectively reduce noise even in a dynamic image by taking advantage of the inter-frame correlation.
As described above, the conventional noise reduction processings using the temporal correlation involve switching to a noise reduction processing mainly using the intra-frame correlation for a dynamic region or actively using the inter-frame correlation by calculating a motion vector and extracting a region having a high inter-frame correlation.